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This paper discusses automatic detection and exploitation 
of structural redundancy in large-scale mathematical programming 
models. From our perspective, such redundancy represents embedded 
special structure which can give significant insight to the model 
proponent as well as greatly reduce solution effort. We report 
experiments with real-life linear programming (LP) and mixed- 
integer (MIP) models in which various methods are developed and 
tested as integral modules in an optimization system of advanced 
design. We seek to understand the modelling implications of these 
embedded redundancies as well as to exploit them during actual 
optimization. The latter goal places heavy emphasis on efficient, 
as well as effective, identification techniques for economic appli- 
cation to large models. Several (polynomially bounded) heuristic 
detection algorithms are presented from our work. In addition, 
bounds are reported for the maximum row dimension of the more 
complex structures. These bounds are useful for objectively 
estimating the quality of heuristically derived assessments of 
structural redundancy. Finally, some additional suggestions are 
made for analyzing nonlinear programming (NLP) models. 



1 . 



INTRODUCTION 



Automatic detection and exploitation of structural 
redundancy in large-scale linear programming (LP) (as well as 
mixed integer programming (MIP) and nonlinear) models has been 
one subject of a continuing research program conducted at the 
Naval Postgraduate School and UCLA over the past decade. This 
exposition draws from various results in that effort, and refers 
(sparingly) to significant work by other researchers. The refer- 
ences contain complete descriptions of these results for the 
interested reader. 

Our scope is intentionally limited to automated methods 
of sufficient efficiency to enable us to economically apply them 
to real-world optimization problems. Thus, we consider only 
those approaches showing greatest promise for immediate practical 
application. Although the interpretations of embedded model 
redundancy can lend profound insights in their own right, we are 
equally interested in detecting errors in data preparation and 
model generation--mathematically mundane issues of fundamental 
importance to the practitioner. 

In this context, our definition of structural redundancy 
includes not only features which permit reduction of effective 
problem size, but also those embedded special structures which 
invite application of special solution methods with enhanced 
efficiency. This somewhat expanded view of redundancy admits 
features which yield to, for instance, basis factorization or 
decomposition . 
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The sheer size of contemporary large-scale LP models 
presents significant computational difficulties, even for dis- 
covery of otherwise elementary structures (in the sense of formal 
complexity) . Implementation of effective structural analysis 
procedures is primarily a matter of exercising large-scale data 
structures efficiently. As we shall see, though, these practical 
considerations can give significant theoretical guidance in the 
specification of efficiently achievable classes of model trans- 
formations . 

That detection of embedded special structure can be of 
practical importance in actual model solution is undisputed. It 
is widely known that explicit simplex operations can be materially 
improved in efficiency by incorporation of basis factorization 
methods (e.g. Graves and McBride [12, 15]). The details of 
such modifications of the simplex procedure are not given here. 
However, the underlying themes of simplex factorization are the 
substitution of logic for floating point arithmetic, and separa- 
tion of the apparent problem monolith into more manageable 
components . 

This work deals primarily with row factorizations. The 
pervasive implied problem for row factorization is the identifi- 
cation of the best embedded structure from all those that may lie 
at hand in any particular model. Conventional wisdom differs as 
to the criterion for this discrimination among factorizations 
of the same class. However, it is generally accepted that the 
row dimensionality of the factorization serves as an excellent 
measure of effectiveness. In this sense, embedded special 
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structures fall naturally into a taxonomy implied by the intrinsic 
complexity of the associated maximum row identification problems. 

We proceed with a discussion of several types of embedded 
special structures detectable by efficient polynomially bounded 
algorithms. These structures are considered in increasing order 
of maximum row identification complexity. We emphasize that 
efficient polynomial algorithms are operationally defined here 
as low-order polynomial in terms of intrinsic problem dimensions 
(e.g. number of rows, columns, and non-zero elements), and not 
in terms of the total volume of model information (e.g. total 
number of bits in all coefficients, ad nauseam). 
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2 . 



OVERVIEW OF THE ANALYSIS 



We are usually faced with the following practical situation. 
An LP (or MIP) is presented, typically in MPS format on magnetic 
tape. Some documentation and possibly solution experience is 
also provided, but the matrix generator and source data elements 
for the model are rarely at hand. The model is frequently sent 
to us because of some difficulty encountered in preliminary solu- 
tion attempts. 

Our efforts are devoted to two issues: Analysis of the 

LP, and solving it efficiently. 

The analysis is initially focused on rather ordinary 
details, primarily establishing computer compatibility, and on 
review of gross problem statistics such as those shown in Table 1 
for the sample problems we will discuss further. At this stage 
we attempt to detect outright disasters before investing more 
time and money. 

For instance, we have frequently found that the actual LP 
bears little resemblance to the intended formulation due to matrix 
generator difficulties, human error, or data base failures. Even 
an LP which seems to follow its formulation template may exhibit 
superfluous objective functions, right hand sides, and so forth. 

After the cursory review, a representative model is selected, 
set up and input to an internal data structure for detailed analy- 
sis. The input process is relatively expensive for large LP models, 
involving conversion of thousands of records. Row and column 
summary statistics are produced and reviewed. At this point, 
pathological coefficient scaling is frequently revealed; this is 
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sometimes caused by poor modeling, and occasionally by bad source 
data elements. For example, we have found at this point that 
unwanted model features (such as, say, production capacity con- 
straints) have been "relaxed” in an ad hoc fashion by providing 
outrageous dummy coefficients (e.g. essentially infinite capacity 
or infinitesimal production rates) . These redundancies, while 
innocently intended, can play havoc with LP solution procedures 
(this is especially true for MIP and nonlinear algorithms) . 

Next, we apply a set of simple reductions to the LP model. 
At this stage, we identify redundancies with two goals. First, 
we want to complete our "bottom-up" analysis of the model. How- 
ever, we also seek to set the stage for actual model solution. 

In this sense, we cannot (ordinarily) afford and we do not 
(routinely) apply at large scale reduction methods which are com- 
putationally equivalent to actual solution of the LP . Rather, we 
discover as much as possible about redundancy in the model by 
efficient, polynomially bounded static analysis not employing 
basis exchanges. From this we attempt to infer qualitatively the 
complete redundancy structure of the model at hand. 

Also, we ignore structural features which have little or 
no bearing on our analysis or solution effort. For instance, we 
are indifferent about degeneracy, since our solution procedures 
exploit this delightful property (Graves [11]). 

Similarly, we do seek certain properties which may make 
our solution procedures much more efficient, but which may not 
be so attractive to analysts using other methods. Among these 
properties are embedded generalized upper bounds (GUB) and net- 
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work, or pure network (NET) rows, for which our interest is 
particularly keen (Bradley, Brown and Graves [6] , Brown and 

Graves [8], Graves and Van Roy [13]). 

Actual solution of the model follows these analysis efforts 
unless we find that the model requires significant modification 
or managerial review. For models that merit solution, we see no 
reason to impose unreasonable restrictions on the model builder. 
That is, the responsibility for efficient solution is ours regard- 
less of the redundancy structure found, as long as this redundancy 
is not in conflict with the intent of the modeler. Thus, all 
reductions must provide an equivalent solution to the model as 
originally posed, and can not require that the model be changed 
or severely modified externally (e.g. requiring general linear 
transformation of the formulation to suit the solver is out of 
the question) . 
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3. DETAILS OF THE ANALYSIS 



For analysis, the linear program is stored in a sparse 
data structure. Nonzero coefficients are stored along with the 
corresponding row index by column with access via column entry 
points. Each row and column has associated with it an external 
label, several coefficient values representing upper and lower 
bounds and ranges, and a coded tag giving the type of constraint 
(e.g. equation) or column and its status in the analysis. 

Although conversion to a super-sparse data structure is 
subsequently required for the solution of the model, the sparse 
structure is much more convenient for analysis of the model 
composition and for model modifications. Very little auxiliary 
storage is required for the analysis and practical problems at 
large scale are routinely analyzed. The system is designed to 
operate on problems with up to 30,000 rows plus columns. (E.g. 
see Bradley, Brown and Galatas [4] for FORTRAN examples using 
these data structures.) 

Our analysis is confined to reductions that do not change 
the feasible region. The analysis can also be called "orthogonal" 
in that the reduction tests are made on the current problem with 
no pivotal transformations actually performed. The reductions 
may show how to transform the problem by removing columns and 
constraints and by the elimination of columns (equivalent to 
pivoting) , but the tests are applied only to the current repre- 
sentation of the problem. 

The analysis is applied to a fully ranged and bounded 
linear program. 



R 



nun 



y .c.x. 

L 3 3 3 



s.t. r. 5 y.a. .X. 2 r 1 Vi (ranged constraints) 
i ‘] i) ] 

Z. < X. 5 u. V j (simple bounds) . 

3 3 3 

Some ranges and bounds may be missing (that is, +°° or ~°°) 
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3.1. Simple Reductions 

Singleton column. If a column has a single nonzero 
coefficient, the column can be removed. The ranges of the con- 
straint that contain the nonzero coefficient are modified to 
construct an equivalent problem. 

Fixed column. If a column has been fixed at a certain 
value, or (equivalently) its upper and lower bounds are equal, 
it can be removed. The ranges of all constraints that have a 
nonzero coefficient in the removed column are modified. 

Vacuous columns. Columns with no nonzero coefficients 
may be removed. The associated variable may assume any value 
between its upper and lower bounds. 

Inconsistent column. Any column with its upper bound 
strictly less than its lower bound indicates that the linear 
program has no feasible solution. 

Free column. A free column results from modeling a 
variable that has no upper or lower bound or by analysis that 
can show that neither the upper nor lower bound is necessary to 
define the feasible region. 

Singleton constraint . If a constraint has a single 
nonzero coefficient, the constraint may be removed. The bounds 
of the column that contains the coefficient are modified. Simple 
bounds are often revealed this way when they are inadvertently 
expressed as explicit constraints. 
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Redundant constraints . A constraint is redundant if 



its removal does not change the feasible region. An examination 
of the bounds on columns with nonzero coefficients yields a test 
for a redundant constraint. For constraint i define 



R 1 = 



l 

a ij >0 



a . . u . + 
ID D 



\ a . . I . 
*i <• 13 3 



and 



R. = 
1 



Y a . . I . + y a..u. 
a. >0 ^ 3 a. <0 ^ 3 

id 



A constraint is redundant if R 1 2 r 1 and R^ > r^. If only one 
inequality holds, the corresponding range can be eliminated (that 
is, set to oo or -«°) . 

Constraints that fix variables . If R 1 = r^ or 
R. = r 1 then each column with a nonzero coefficient in the con- 

l 

straint must be fixed at the appropriate bound in order for the 
constraint to be satisfied. The constraint can then be removed. 

Inconsistent constraints. If R 1 < r^ or R^ > r 1 , 
then the LP has no feasible solution. 



Vacuous constraints. Constraints with no nonzero 
coefficients may be removed. If r 1 < 0 or r^ > 0 , then the 
LP has no feasible solution. 

All of the tests above are applied in a single pass. 
Since the reductions (if any) may make it possible to identify 
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new reductions, the complete analysis consists of repeated passes 
until no additional reductions are found. Table 2 displays the 
results for the sample problems. The times given are for execution 
on an IBM 360/67 using FORTRAN H (Extended) without code optimi- 
zation. 

With real-life LP (and MIP) models, a remarkably large 
fraction of model constraints can be removed by these simple 
techniques. For some cases, models have been nearly solved this 
way. 

We have often been surprised at the number of reduc- 
tions achieved after several passes. The repeated passes can 
serve to unravel a model and reveal special structure that is 
quite obscure in a static analysis. 

Experiments with some of these reductions have been 
reported by Brearley, Mitra and Williams [7]. More extensive 
work at large scale has been done by Bradley, Brown and Graves 
[5] and by Krabek [14]. 
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3.2. Transformation Reductions 

Equations confine the feasible region to an affine 
subspace of lower dimension. It is possible to eliminate an 
equation and a column with a nonzero coefficient in the equation. 
If a^ 7* 0 in an equation constraint i, then 



X. = (r 1 - T a. .X.)/ 
k ^ 1 



3 ' T 

ik 



can be used to eliminate X^ from the problem. The equation is 
eliminated but the bounds on generate a constraint 



“k s <ri - a i: x : )/a ik s \ • 



This elimination is equivalent to pivoting on a ^ . Although 
repeated application will remove all equations, this will not in 
general make the problem easier to solve or reduce the number of 
constraints . 

There are two special cases where the elimination of a 
column can reduce the number of constraints. If the column X^ 
has no upper or lower bound (i.e. a "free" variable) , then the 
generated constraint is redundant and thus may be eliminated. 

If the equation has only two columns with nonzero 
coefficients, the generated constraint is a simple bound on a 
variable and thus may be combined with the existing bounds on 
the variable. 

A particular type of constraint that is common to many 

v 

models leads to an equation that has a nonzero coefficient for 
a column with no bounds. Commonly called material balance equations , 
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these constraints set one nonnegat'Lve column (i.e. with infinite 
upper bound) equal to the sum of several other nonnegative columns. 




0 < X. 



0 < X_. , j € J . 



Since the Xj j € J are all restricted to be nonnega- 



tive, it is easy to see that the bound X^ > 0 is redundant — that 
is, if it is eliminated the feasible region is unchanged. Thus 
k can be regarded as a column with no bounds and the elimination 
of X ^ will reduce the number of constraints by one. 



ance equation results in removing the bound from X^ , when 
eliminating X^ any single equation that has a nonzero coeffi- 
cient in column k can be substituted out. 

The analysis applies to a generalization of material 
balance rows : 



where a.. > 0, b. and Z . > 0 and a. . < 0 V j€J. Although 
IK i 3 13 

this is not the most general form which can be used to designate 
that the bound (s) on X^, are redundant, this form (and its nega- 
tive) captures all the cases in the real-world problems that we 
have analyzed. 



Notice that although the analysis of the material bal- 





0 - V - x j - u j 
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Doubleton equations . Equation constraints with exactly 
two nonzero coefficients can be identified for immediate elimina- 
tion. The elimination of the equation and one of two columns is 
accomplished as described above, the generated inequality contains 
just a single nonzero coefficient so it can be removed by modify- 
ing the bounds on the surviving column. 

Free column equations . An equation with a nonzero 
coefficient for a free column can be identified for immediate 
elimination along with the free column. The transformation to 
remove the constraint and column is equivalent to pivoting on 
the nonzero coefficient. 

Redundant bounds. Our analysis uses only the generalized 
material balance equation to identify columns that have no bounds. 
It is possible to generalize this idea to include the use of all 
constraints to identify such columns. IJach constraint may be used 
to generate bounds on all the columns with nonzero coefficients. 

For constraint i with a.. >0 



7 a . . u . 

a..>0 3 

ID 



a . . <0 
ID 



a . . % . + r. S a . .. X. 
lj ] l lk k 



Sr 1 - l a . . £ . - 7 a..u. 

a. >0 13 3 a. <0 13 3 
ID ID 

D^k 



An analogous result can be constructed for a^ < 0. 

The intersection of these bounds from all the constraints 
infer bounds for column k which may reveal and u^ to be 
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redundant and thus permit k to be designated a free column, 
or which can be used to tighten and u^ . 

Although this test can be done efficiently for any 
particular a^, there are potentially quite a large number of 
candidates to test. We have not included this test in our results 
here, and we further believe that there are few real LP problems 
for which this test yields significantly more columns without 
bounds than examination for generalized material balance equations. 

One particular situation for which this test may be 
selectively applied is that in which a coefficient is much larger, 
or much smaller than its cohorts. In this case the inferred bounds 
may reveal infinitesimal or (respectively) gigantic bounds for 
X^, possibly suggesting deletion of the column. 

Table 2 shows doubleton equations found after the finaj. 
simple reduction pass. Note the last model (ODSAS) for which 
almost all constraints are identified as doubleton equations. 

A sample analysis for material balance equations per- 
formed on the PAPER model detected 1,645 such constraints. 

It is not always obvious whether actually applying a 
particular transformation reduction will produce an LP model 
which is easier to solve. In particular, transformation reduc- 
tions can generate a "reduced, equivalent LP" which is actually 
denser, and not necessarily as well-scaled as its progenitor. 

On the other hand, some reductions offer a decided 

advantage for solution efficiency. For constraints like 

X.-X, =0, X . , X, > 0, both variables must be in the basis for 

j k j k 

them to assume a positive value. For many commercial linear 
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programming systems, partial pricing and the lack of effective 
mechanisms to cope with degenercy do not allow the efficient 
treatment of the special relationship between the variables. 
Similarly, for material balance equations with nonnegative vari- 
ables, at least two variables with coefficients of opposite sign 
must be in the basis before any of the variables can assume a 
positive value. The transformation reductions eliminate these 
particular instances where relationships among variables inter- 
fere with the solution progress. 

For large-scale models, we analyze the reductions care- 
fully, using all information available for the model and the 
problem it addresses. Numerical and structural consequences of 
reductions are critically reviewed in concert with the algebraic 
interpretations and modeling discoveries which they characterize. 

Notice that both simple and transformation reductions 
may be viewed as linear operators that do not change the feasible 
region of the problem. After the reduced problem is solved, the 
inverse operators applied to the optimal solution construct an 
optimal solution to the original problem. 

The analysis makes no special use of an objective func- 
tion. Nonbinding constraints (that is, "free rows") may be 
included in the problem. The objective function and any free rows 
designated to be included are only examined in determining if a 
column has a single nonzero coefficient. Thus the reduction can 
be done for several different objective functions simultaneously. 
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3.3. Generalized Upper Bounds 

Rows for which each column has at most one non-zero 
coefficient (restricted to those rows) collectively form a gen- 
eralized upper bound (GUB) set. Usually, we additionally require 
that the coefficients in these rows be capable of being rendered 
to ±1 by simple row or column scaling. 

The problem of identifying a GUB set of maximum row 
dimension is NP-hard, making optimal GUB identification algo- 
rithms hopelessly inefficient for our purposes. Heuristics 
adapted from work by Graves and by Senju and Toyoda [13] (see also 
Brearley, Mitra and Williams [7]) work very effectively and depend- 
ably at large-scale to find maximal GUB sets. 

Unfortunately , the problem of determining just the size 
of the maximum GUB set is also NP-hard. However, Brown and Thomen 
[9] have developed bounds on the size of the maximum GUB set which 
are sharp and easily computed. These bounds have been used to 
show, in some cases, that maximum GUB sets have been achieved via 
heuristic methods. In any case, the bounds provide an excellent 
objective measure of the quality of any GUB set, regardless of 
the means of its derivation. Frequently, manual GUB analysis 
will suffice for models with amenable structure. 

The bounds are developed in terms of the number of dis- 
tinct conflicts present in the model. Two rows are in conflict 
if they each have a non-zero element in a common column, making 
them mutually exclusive in a GUB set. If s^ is the number of 
rows in conflict with row i, then the total problem conflict 
count for a model with m rows is 



19 



GUB set is 



c = j [ s. < m(m-l) . 
i 

A problem- independent bound on the size of the maximum 

u^ = |_ . 5 + /. 25 + m(m-l) - 2c , 

where J_ indicates truncation to the next lower integer. 

A tighter, problem-dependent bound is 

. m-l" ^ » c 5 (m-y)y 

' j_ . 5 + /. 25 + y (2m-y-l) - 2c , c > (m-y)y ; 



where 



y = max s . , 

i 

and [~ indicates rounding to the next higher integer. 

Tighter upper bounds have been derived for the size of 
the maximum GUB set, as well as lower bounds. 

Table 3 contains the results of automatic GUB identifica- 
tion applied to the benchmark models. Constraint eligibility is 
based on the capability to scale the row to contain only 0, ±1 
coefficients. GUB quality is the number of GUB rows found, ex- 
pressed as a percentage of the best known upper bound on maximum 
GUB row dimension (actual GUB quality may be better than this 
conservative estimate) . The results were obtained using FORTRAN H 
(Extended) with code optimization. 
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3.4. Implicit Network Rows 

Implicit network rows are a set of rows for which each 
column has at most two non-zero coefficients (restricted to those 
rows) and for which columns with two non-zero coefficients (in 
those rows) can be converted by simple row and column scaling 
such that the non-zero coefficients have opposite sign. Such 
rows in LP are commonly called networks with gains. 

Pure network rows (NET) can be converted by simple row 
and column scaling such that all non-zero coefficents (restricted 
to those rows) have value ±1, and such that columns with two 
non-zero coefficients (in those rows) have one +1 and one -1. 

Such rows in LP are called pure networks. 

Simple row and column scaling is restricted such that 
application of each scale factor renders an entire row, or column, 
to the desired sign (and unit magnitude for pure NET) . 

The problem of identifying a NET factorization of maximum 
row dimension is NP-hard (Wright [18]), making optimal NET identifi- 
cation algorithms unattractive in a practical sense. The problem of 
determining just the size of the maximum NET set is also NP-hard. 
Thus, heuristic identification methods are mandated. 

An extension of GUB heuristics can be used to achieve 
NET factorizations. First, a GUB set is determined by methods 
mentioned in Section 3.3. Then, a second GUB set is found from 
an eligible subset of remaining rows. The second GUB set is con- 
ditioned such that its row members must possess non-zero coeffi- 
cients of opposite sign in each column for which the prior GUB 
set has a non-zero coefficient. 
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This double-GUB (DGUB) identification yields a bipartite 
NET factorization. Thus, DGUB heuristically seeks the maximum 
embedded transportation or assignment row factorization. Pure 
network equivalents derive from proper editing of eligible rows. 

Generalizing on the theme of Senju and Toyoda , a 
more general method has been developed by Brown and Wright f 10 ] 
for direct factorization of implicit network rows. Pure 
NET rows are identified with the same procedure by simple 
screening of admissible candidate rows. 

This heuristic is designed to perform a network factori- 
zation of a signed elementary matrix (0, ±1 entries only) . It is 
a deletion heuristic which is feasibility seeking. The measure 
of infeasibility at any point is a matrix penalty computed as 
the sum of individual row penalties. The algorithm is two-phased, 
one pass, and non-backtracking. The first phase yields a feasible 
set of rows, while the second phase attempts to improve the set 
by reincluding rows previously excluded. Each iteration in Phase 
I either deletes a row or reflects it (multiplies it by -1) and 
guarantees that the matrix penalty will be reduced. Thus, the 
number of iterations in Phase I is bounded by the initial value 
of the matrix penalty, which is polynomially bounded. 

Let A = be an mxn matrix with a^ = 0,±lVi,j. 

Problem: Find a matrix N = [n^j] with (m-k) rows and n 

columns which is derived from A by 
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1. Deleting k rows of A where k > 0, 

2. Multiplying zero or more rows of A by -1, 
where N has the property that each column of N has 
at most one +1 element and at most one -1 element. 

We wish to find a "large" N in the sense of containing as many 
rows as possible, i.e. minimize k. 

Terminology and Notation: 

1. E is the set of row indices for rows eligible for inclusion 
in N and is called the eligible set. 

2. C is the set of row indices for rows removed from E in 
Phase I (Deletion) . Some rows in C may be readmitted to 
E in Phase II. C is called the candidate set. 

3. The phrase "reflect row i' of A" means to multiply each 
element in row i ' by -1, i.e. a i • j ^ -a i'j ^ D* 

4. Other notation will be defined in the algorithm itself. 

ALGORITHM: 

Phase I - Deletion of Infeasible Rows 

Step 0: Initialization. Set E = {l,2,...,m}, C = <f> . 

For each column j of A compute the + penalty (kt) 
and the - penalty (K ^ ) as follows: 

Kj = ( l 1 ) - 1 , K? = ( l 1 ) - 1 . 

J i€E : a . .>0 J i€E:a. . <0 

ID ID 

These penalties represent the number of excess +1 and -1 
elements, respectively, in column j which prevent the rows 
whose indices remain in E from forming a valid N matrix. 
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A penalty value of -1 for kT (K ^ ) indicates that the 
column does not contain a +1 (-1) element. 

Step 1: Define Row Penalties. For every i €E, compute a row 

penalty (p^) as follows: 

Pi = I K j + l k" 
j : a . . >0 J j : a . . <0 -* 

ID ID 

This is simply the sum of + penalties for all columns in 
which row i has a +1 plus the sum of - penalties for 
all columns in which row i has a -1. 

Step 2: Define Matrix Penalty. Compute the penalty (h) for 

the matrix by summing the row penalties as follows: 

h = l Pi • 

i<EE 

If h = 0, then go to Step 7. Otherwise, go to Step 3. 

Step 3: Row Selection. Find the row i' €E with the greatest 

penalty, i.e. 

Find i' €E such that p., = max p. . 

1 i€E 

(If there is a tie, choose i' from among the tied values.) 
Compute the reflected row penalty p^ , for i' as follows: 

Pi. = I (KT+1) + l <K%1) . 

j : a i ' j >0 J j :a i.j <0 
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This would be the row penalty for row i' if it were to 
be reflected. 

Step 4: delete y or Refleet Row. 

Case i) jj^, > p ±I . Let E«-E-{i’},C«-CU{i'}. Go 
to Step 5. 

Case ii) p^, < p^ , . Reflect row i'. Go to Step 6. 

Step 5: Reduce column penalties as follows: 

For all j such that a. , . >0, K - !" +■ k1" 

J I’D 3 D 

For all j such that a. , . < 0, K . •*- kT 

J ID D D 

Go to Step 1. 

Step 6: Change column penalties as follows: 



Using the 


a. , . 
i D 


values 


after 


reflection of 


row i 1 


1 

t 




For all j 


such 


that 


a. , . > 
i'D 


0 , 


K + + K + + 1 
D D 


and 


k7 

D 


«- k 7 - l 
D 


For all j 


such 


that 


a . , . < 
ID 


0 , 


Kt K + - 1 

D D 


and 


k" 

D 


<- k 7 + l 
D 



Go to Step 1. 



Phase II - Reinclusion of Rows from C 

Step 7: Eliminate Conflicting Rows. The rows with indices in E, 

some possibly reflected from the original A matrix, form a 
valid N matrix. However, some of the rows removed from E 
and placed in C may now be reincluded in E if they do not 
make h > 0. Remove from C (and discard) all row indices 
for rows which, if reincluded in E in present or reflected 
form, would make h > 0. 



- 1 . 

- 1 . 
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i . e . 



Remove i from C if 



a) 


3 h 


such 


that 


a . 
^i 


> 


0 


and 


K + 

3l 


= 0 , 








or 


a . . 
^i 


< 


0 


and 


k7 

3 1 


= 0 ; 




and 




' 














b) 


3 j 2 


such 


that 


a. . 

il 2 


> 


0 


and 


k7 

32 


= 0 , 








or 


a. . 
^2 


< 


0 


and 


K + 

3 2 


= 0 . 



If C = 4>, STOP, otherwise go to Step 8. 

Step 8: Select Row for Reinclusion. At this point a row from 

C may be reincluded in E. There are several possible 
schemes for selecting the row. After the row is reincluded, 
the column penalties are adjusted. Then go to Step 7. 

No dominating rule has been discovered for breaking ties 
in maximum row penalty encountered in Step 3. The rule used for 
the computational results presented herein is to select the row 
with the minimum number of non-zero entries in an attempt to place 
a larger number of non-zero entries in the network set. Other 
possible rules are "first-come, first-served," maximum number of 
non-zero entries, type of constraint, or modeler preference. 

Modifications can be made to Step 0 to allow for 1) 
Matrices including non- 0, ±1 entries and/or 2) Pre-specif ied 
network rows. The modifications are: 
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1 . 



E = {i | a i j = 0 /± 1 for all j } , 

2. Let P = { i | row i is prespecified} , 

E X- E - P . 

After computation of K - !" and K. find for all j : 

1 3 > 



if 


3i€P 


such 


that 


a. . = 1 then 
13 


K + 

3 


- K + 
3 


+ 1 i 


if 


3i€P 


such 


that 


a . . = -1 then 
ij 


k7 

3 


k7 

3 


+ 1 . 



At termination of the algorithm, the rows in N are given by 
E U P. 

One easily obtained upper bound on the maximum row 
dimension of the network factorization is: 

u 1 = m - MAX(K + +K7) . 
j 3 3 

This bound is easily computed and evidently sharp. It 
can be used to objectively evaluate the quality of a heuristically 
derived network factorization. The bound may also be used to 
preemptively terminate identification effort. 

Another, generally tighter bound has been developed by 
Wright [19] which is based on the reflection and deletion potentials 
for each row in the eligible set. Using this information it 
is possible to obtain a lower bound on the number of rows which 
must be deleted to achieve a feasible network set. The upper 
bound is then: 

«2 = m ” lower bound on rows deleted. 
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This bound is also evidently sharp and is the bound 
used to compute NET quality in the following table. 

Table 4 displays the results of DGUB and NET factoriza- 
tions of the benchmark models. Row eligibility is determined by 
the capability to scale each row, by row scaling alone, to con- 
tain only 0, ± 1 entries. The NET quality is the number of NET 
rows found, expressed as a percentage of the upper bound on maxi- 
mum NET row dimension given above (actual NET quality may be 
considerably better than this estimate) . 
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3.5. Hidden Network Rows 

Hidden network rows 1 are a set of rows which satisfy 
NET row restrictions after full linear transformation of the 
model. That is, realization of these (LNET) rows may require a 
general linear transformation of the original model. 

The discrimination between implicit and hidden network 
rows is not (necessarily) in their use, but rather in their 
detection. The transformation group associated with implicit 
network rows involves only permutations and simple scaling of 
individual rows and columns. The hidden network rows require a 
completely general linear transformation and partial ordering. 
Thus, identification of hidden networks requires significant 
computation just to identify eligible rows, since any given row 
may conflict with subsets of its cohorts after transformation. 

This problem has been solved for entire hidden network 
factorization, where all rows are shown to be LNET or the algo- 
rithm fails. Bixby and Cunningham [2] and Muslem [16] have given 
polynomially complex methods for entire LNET conversion. (The 
entire GUB problem is polynomial as well.) 

Strategically, the entire hidden LNET factorization 
requires two steps: 

DETECTION : necessary conditions for existence of an entire 

LNET factorization must be established, and 
SCALING: a linear transformation to achieve the NET 

structure must be determined, if one exists. 

* 4 * , , • 

We have coopted the term hidden from Bixby [1], but his defini- 
tion may not superficially appear to be equivalent. 
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Cunningham and Bixby attempt detection, followed by- 
scaling. Musalem tries scaling, then detection. This is a cru- 
cial difference between methods, since problems which can not be 
completely NET factorized may fail in either step. 

Briefly, Cunningham and Bixby detect by showing that 
the incidence matrix of the model rows can be converted to a 
graphic matroid. They employ a method by Tutte (see references 
of Bixby and Cunningham. Given success, the graphic record 
of the detection is used to attempt to scale the model to NET, or 
to show that no such scaling exists. 

Musalem scales the model to a ±1 matrix, and then uses 
a method by Iri (see references of Musalem) to build a tree, 
edge by edge, which reveals the partial ordering coincident with 
entire hidden LNET factorization. 

Both methods are polyncmially complex. However, 
entire LNET factorization is relatively expensive by either method 
in that quite a large amount of real arithmetic and logic is 
required. Underlying data structures have not been suggested 
for either method. Both methods fail if complete LNET factoriza- 
tion is impossible, and neither leaves the investigator with much 
information useful in salvaging a partial LNET factorization. We 
conjecture that risk of preemptive failure narrowly favors the 
Musalem approach, since he defers the relatively involved 
detection step. 

Locating a hidden LNET factorization of maximal row 
dimension has been suggested by Bixby [1] and by Musalem, 
but no concrete method is given and no computational testing is 
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reported. Evidently, the maximum LNET problem is NP-hard, and 
its maximal relaxation remains unsolved in the practical sense 
of this report. 
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4. EXTENSIONS TO MIXED INTEGER AND NONLINEAR MODELS 



Mixed integer (MIP) and nonlinear (NLP) optimization 
models present additional challenges, especially at large scale. 

Our interest in the detection of structural redundancy is inten- 
sified since general purpose algorithms for (MIP) and (NLP) nor- 
mally operate by solving sequences of many embedded LP models. 

This provides added impetus to the analysis of problems prior to 
their actual solution, and economically justifies some additional 
initial investment in problem analysis. 

4.1. Mixed Integer Extensions 

The structural analysis presented for LP is also appli- 
cable to MIP. Since most real-world models and many commercial 
optimization systems have only binary variable capability, our 
analysis addresses binary variables exclusively. Thus, binary 
expansion of integer variables is a prerequisite. We also 
assume that reductions requiring scaling of binary columns are 
inadmissable . 

When a reduction tightens a bound for a discrete variable 
the bound is rounded to the nearest integer (down for u^ and up 
for &j). Any tightening of bounds for a binary variable immedi- 
ately results in a fixed column or an inconsistent column. The 
reductions for fixed columns, vacuous columns, inconsistent 
columns, singleton constraints, vacuous constraints, constraints 
that fix variables, and redundant constraints are applied exactly 
as described for LP. 

The treatment of doubleton equations requires special 
consideration to identify inconsistent constraints. 
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A doubleton equation with both columns binary has 
either one solution, two solutions or no solutions. All four 
possible solutions (0,0), (1,0), (0,1) and (1,1) are tried. 

If only one solves the equation, the binary variables 
are fixed at these values and the constraint is removed (the 
test for constraints that fix variables will also discover the 
equation with a single solution and accomplish the same reduction) . 

For the case with two solutions there can only be two 

situations: consider the doubleton as a..X. +a. 0 X- = b. with 

ll 1 i2 2 i 

a il ^ 0 and a^ ^ 0* Then if (0,0) and (1,1) can both 
solve the equation, this implies that b = 0 and a^ = -a^. 

If (0,1) and (1,0) are solutions, this implies that 
a^ = a^ = b. Both cases are treated correctly by the trans- 
formation described for continuous variables. 

If there are no solutions, the constraint is designated 
as inconsistent. 

For a doubleton equation with one continuous and one 
binary variable, the transformation described for continuous 
variables is used, but it must be the continuous variable that 
is eliminated. 

The reductions to eliminate singleton columns and to 
designate a free column equation may not be applied to binary 
variables. This is ensured by marking the binary columns as 
ineligible for these reductions. Note that since a binary vari- 
able can never be designated as a free column, binary variables 
cannot be eliminated as a variable in a free column constraint 
but may be among the other columns in such a constraint. 
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The redundant bounds test may be used to tighten bounds 
on binary variables and thus fix them, or show that the MIP has 
no feasible solution. 
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4.2. Nonlinear Extensions 



Large-scale nonlinear optimization, though not yet in 
wide use, can benefit from the analysis techniques given here 
for LP, and demands some additional special treatment. 

We have experience with only two large-scale, general- 
purpose optimization systems with full nonlinear capability: our 

own X-system and MINOS/ Augmented [17]. Both of these systems 
can accept linear problem features and labels in MPS format and 
nonlinear terms from function generators. Both systems can also 
employ several alternate problem generation interface standards. 

These systems are each designed to exploit any linearity 
or near-linearity in the NLP. Given a starting solution, it is 
of no little interest to analyze the linear portion of the NLP 
which will, after all, be solved many times — the reason that 
we support al l LP features for NLP. Also, any local lineariza- 
tion of the NLP is subject to analysis. 

However, it has been our misfortune to have repeatedly 
discovered that NLP presents us with unique structural curiosities. 

We refer to the foremost among these as function coordi- 
nation. There are myriad opportunities with NLP to unwittingly 
introduce discontinuities and miscellaneous unruliness in functions 
and derivatives. Whether by programming error, mathematical 
blunder or numerical difficulty, these errors inflict great 
vexation and expense. 

Detection of such difficulties is quite challenging 
since, unlike LP, procedures and data are used to express the 
problem at hand. As a bare minimum, we employ a preemptive analysis 
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module that acts as a complete surrogate for the optimizer, 
employing standard interface conventions and exercising all 
functions and data. 

The starting solution and scaling parameters are used 
to check analytic gradients (if supplied) with numerical differ- 
ence approximations. Approximation of functions is then attempted 
to reveal behavior local to the initial solution such as apparent 
convexity and degree of nonlinearity. Optionally, the first step 
of the algorithm is simulated and the same analyses performed. 

From initial results algorithm tolerances may be changed, 
programming errors detected, and so forth, until acceptable model 
behavior is observed. 

In some cases, suspicious functions may be evaluated 
at column bounds to see if numerical arithmetic faults occur. 

Some models require construction and maintenance of a trust region 
for the approximations implied by the NLP algorithm, and prior 
analysis is absolutely essential in these cases. 

Structural analysis of MIP and NLP can frequently — 
even repeatedly--presage outright failure of the solution algo- 
rithms to be employed. For these models, the effects of struc- 
tural redundance can be far more significant than for simple LP. 
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5. CONCLUSION 



The techniques reported here have been used with great 
success on a wide variety of large LP (MIP) models. The context 
of this research is somewhat atypical in that the models which 
we work with are often sent to us for analysis and solution pre- 
cisely because they have already failed elsewhere. In these 
cases, our motives are to quickly diagnose suspected trouble 
before optimization, prescribe remedies, and perform the actual 
optimization reliably and efficiently. 

This has undoubtedly biased our view of structural detec- 
tion methods. Practical considerations arising from turnaround 
deadlines and the specific advantages of our own optimization 

■j* 

system (Brown and Graves [8]) have colored our judgment. Many 
provocative suggestions for further research have not been pursued, 
either due to lack of opportunity, to poor intuition, or to simple 
economics. Whether or not by equivalent prejudice, Krabek [14] 
reports some similar methods for detecting redundancy in large- 
scale MIP. 

Various commercial optimization systems support "CRASH," 
"REDUCE," and other operators which implement some of these reduc- 
tions automatically during LP solution. These systems are not 
reviewed here. We stress the value of structural analysis 



+ 

The X- sy stem (XS) differs in many ways from classical large- 
scale mathematical programming systems; it simultaneously sup- 
ports simple and generalized upper bounds, general basis fac- 
torization, MIP, nonlinear, and decomposition features. In 
addition, the fundamental LP algorithm has been enhanced to 
intrinsically incorporate elastic range restrictions. XS is 
particularly suited for solution in limited time of large models 
with complicating features. 
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techniques as stand-alone tools, rather than as exclusive 
features of actual LP-solution algorithms. 

A great deal of insight has been gained from these experi- 
ments. The cost of analysis is truly insignificant relative to 
the information and solution efficiency gained thereby. Revela- 
tions have ranged from outright rejection of absurd formulations, 
to subtle inferences on the project management and inter-personal 
relations among model proponents. Very few models fail to reveal 
some totally unsuspected structural curiosity. Indeed, it is 
often some small aberration that proves most revealing. Sometimes, 
the combined effects of several minor features collectively con- 
tribute to a discovery of significant model attributes. 

Our general operational guideline has been to avoid heavy 
computational investment in model analysis. Rather, highly 
efficient methods are used veveatedlv on variations of each model. 

L 

Manual and intuitive analysis of these results usually reveal 
much more than could be reasonably expected from any totally auto- 
mated method of exponential complexity. After all, just the names 
of rows and columns can be expected to reveal a great deal about 
the model, but exploiting this mathematically virtually defies 
automation in any general sense; interactive analysis of large- 
scale models is uncompromisingly challenging in a technical sense 
and equally rewarding. 

Large degrees of structural redundancy are routinely found 
as intrinsic features in real-life models. However, we feel that 
it is an abominable practice to proselytize in favor of some par- 
ticular model structure at the expense of model realism or common 
sense . 
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For instance, network models have recently received 
unprecedented attention in the literature. The implication has 
often been that since networks are usually found in models, net- 
works should be used as the exclusive model. This is, of course, 
patent nonsense, smacking of a solution in search of a problem. 

An analyst should view intrinsic redundancy as an interesting 
feature of models, rather than forcing models to exhibit minimal 
redundancy, or requiring that they follow some particular struc- 
tural pattern. 

As for automating the discovery of all redundancy in a 
model, this exercise seems to be almost exclusively academic with 
large-scale, real-life LP projects. In those rare cases for which 
such extensive analysis is justified, we suggest a straightforward 
view and a frontal attack with an imbedded LP optimizer. 
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